Method and apparatus for increasing an amount of memory on demand when monitoring remote mirroring performance

ABSTRACT

A method and storage system for increasing an amount of memory in a queuing area on the storage system. The storage system includes first and second storage subsystems connected to each other via a path. A primary volume in the first storage subsystem and a remote secondary volume in the second storage subsystem are mirrored and operated in the asynchronous mode. A queuing area having memory is provided in the second storage subsystem for temporarily storing data transferred to the second storage subsystem from the first storage subsystem in response to a write input/output (I/O) issued by a host to write data in the primary volume. Data temporarily stored in the memory is retrieved and stored in the remote secondary volume. An unused area of the queuing area is monitored and the memory increased if the unused area becomes less than a predetermined amount.

CROSS-REFERENCES

This is a continuation application of U.S. Ser. No. 11/360,470, filedFeb. 24, 2006 (now U.S. Pat. No. 7,136,983), which is a continuationapplication of U.S. Ser. No. 10/784,260, filed Feb. 24, 2004 (now U.S.Pat. No. 7,039,785).

BACKGROUND OF THE INVENTION

The present invention relates generally to techniques for implementingthe remote mirroring function. More particularly the present inventionrelates to a method and apparatus for increasing an amount of memory ondemand in a storage system implementing the remote mirroring functionwhen the mirrored pair of volumes of the storage system is operated inthe asynchronous mode.

Conventionally, there have been two types of approaches to storage-basedvolume replication, namely local replication and remote (copy)replication. Both technologies mirror files, file systems, or volumeswithout using host CPU power. When a host conducts a host input/output(I/O) such as a write I/O of data to a primary volume (PV) of a storagesystem, the storage system automatically copies the data to areplication (secondary) volume (SV). This mechanism ensures that PV andSV are identical.

Local replication duplicates the primary volume within a first storagesystem, so that when the host writes data to the PV, the first storagesystem also stores the data to a local secondary volume (LSV). Localreplication is typically used for taking backups.

Remote replication duplicates volumes across two or more storage systemsso that when the host writes data to PV, the first storage systemtransfers the data through paths, such as ESCON, Fibre Channel, T3,and/or IP networks, to at least one second storage system for storage ina remote secondary volume (RSV) included therein. Remote replication istypically used to enable the recovery of data from disasters, such asearthquake, flood, fire, and the like. Even if the first storage systemor the whole data center at the primary site is damaged by a disaster,data is unaffected at the secondary site and business can be resumedquickly.

There are at least two modes of transferring data to implement remotemirroring between local and remote storage systems, namely synchronousmode and asynchronous mode. In the synchronous mode, all write I/O's tothe PV of the first storage system are mirrored at the RSV of the secondstorage system. In the asynchronous mode, in response to a write I/O,the first storage system completes the write I/O and then asynchronouslytransfers the write data to the second storage system for storage on theRSV. Specifically, the write data to be copied to the RSV of the secondstorage system is temporarily stored in a queuing area, such as cachememory, disk storage, Non-Volatile Random Access Memory (NVRAM) etc. Thewrite data is retrieved from the queuing area and then stored in the RSVof the second storage system.

The queuing area where the write data is temporarily stored may run outif the write I/O rate (write I/O's per second) to the primary volume ofthe first (local) storage system is much higher than the throughput forcopying the write data from the queuing area to the remote secondaryvolume of the second (remote) storage system. Once the unused area ofthe queuing area becomes less than a certain percentage of total area ofthe queuing area, the storage system mirroring function will initiate athrottling function to curb the new write I/O's. Thus, the performanceof the storage system with respect to write I/O's degrade over time.

A current proposed solution is that if a customer experiences adegradation in performance with respect to write I/O's, then thecustomer calls a customer support center which analyzes the cause of theproblem and may increase the capacity of the queuing area if necessary.The total turn around time for this proposed solution if long cannegatively affect customer satisfaction. This negative affect oncustomer satisfaction is more likely to occur since the turn around timeis influenced by many human interactions that are necessary.

SUMMARY OF THE INVENTION

The present invention provides a method and apparatus for increasing anamount of memory on demand in a storage system implementing the remotemirroring function when the mirrored pair of volumes of the storagesystem is operated in the asynchronous mode.

According to the present invention a method and storage system isprovided for increasing an amount of memory on demand in the storagesystem implementing a remote mirroring function. The storage systemincludes first and second storage subsystems which are connected to eachother via a path and the first storage subsystem is connected to a host.A primary volume is provided to the first storage subsystem and a remotesecondary volume is provided to the second storage subsystem. The remotesecondary volume is a copied volume of the primary volume and theprimary volume and the remote secondary volume are operated in theasynchronous mode. A memory is provided in the second storage subsystemfor temporarily storing data transferred to the second storage subsystemfrom the first storage subsystem in response to a write input/output(I/O) issued by the host to write data in the primary volume. Datatemporarily stored in the memory is retrieved and stored in the remotesecondary volume. An unused area of the memory is monitored and theamount of the memory is increased if the unused area becomes less than apredetermined amount.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and a better understanding of the present invention willbecome apparent from the following detailed description of exampleembodiments and the claims when read in connection with the accompanyingdrawings, all forming a part of the disclosure of this invention. Whilethe foregoing and following written and illustrated disclosure focuseson disclosing example embodiments of the invention, it should be clearlyunderstood that the same is by way of illustration and example only andthe invention is not limited thereto, wherein in the following briefdescription of the drawings:

FIG. 1 illustrates the configuration of a system for increasing anamount of memory on demand in a storage system according to the presentinvention;

FIG. 2 illustrates an example of a disk management table for managingdisk drives to implement the present invention;

FIG. 3 illustrates an example of a cache management table for managingcache memory to implement the present invention;

FIG. 4 illustrates an example of a monitoring result table for storingthe results of monitoring the resources of a storage system according tothe present invention;

FIG. 5 is a flowchart illustrating an example of a process forincreasing an amount of memory on demand in a storage system accordingto the present invention; and

FIG. 6 is a flowchart illustrating an example of the details ofprocessing performed to implement Step 503 of FIG. 5 according to thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

As per FIG. 1, the present invention provides a storage system whichincludes a first storage subsystem 101 and a second storage subsystem102 that are connected to each other by a path 105 and implements aremote mirroring function. The method and storage system of the presentinvention as provided increases an amount of memory on demand in thestorage system as needed depending on the state of a queuing area.

As per FIG. 1, the first storage subsystem 101 includes a primary volume(P) 103, a storage controller 106-1 which includes a cache managementtable 190-1, a disk management table 191-1 and a resource/licensemanager 195-1, and a queuing area 196-1. The first storage subsystem 101is connected to a host 140 and to a network 180.

Further, as per FIG. 1, the second storage subsystem 102 includes aremote secondary volume (S) 104, a storage controller 106-2 whichincludes a cache management table 190-2, a disk management table 191-2and a resource/license manager 195-2, and a queuing area 196-2. Theremote secondary volume (S) 104 is a copied volume of the primary volume(P) 103 and the primary volume (P) 103 and the remote secondary volume(S) 104 are operated in the asynchronous mode. The second storagesubsystem 102 is connected to hosts 107 and 108. The second storagesubsystem 102 and the hosts 107 and 108 are connected to the network 180via a local area network (LAN) 197.

As illustrated in FIG. 1 the queuing area 196-1 and 196-2 of eachstorage subsystem 101 and 102 can for example be cache memory 1960-1,1960-2, disk drives 1961-1, 1961-2, or memory devices 1962-1, 1962-2.Any of these devices would be acceptable and of course such devicescould, for example, be pre-installed and configured so as to permitexpansion of its storage capacity.

According to FIG. 1 a management center system 150 is provided formanaging the storage resources of the storage system and variouslicensing matters with respect to the storage system including variouslicensing keys as will be described below. The management center system150 is connected to the network 180 and thereby to the first storagesubsystem 101 and the host 140 via the network 180 and to the secondstorage subsystem 102 and the hosts 107 and 108 via the network 180 andthe LAN 197. The management center system 150 includes a storage 198which serves as a repository of information collected from theresource/license managers 195-1 and 195-2, each of which manages theresources of its respective storage subsystem 101, 102 based on thecache management table 190 and the disk management table 191.

FIG. 2 illustrates an example of a disk management table 191 formanaging disk drives to implement the present invention. The diskmanagement table 191 includes a plurality of entries, wherein each entryincludes at least a device number field 201, an installation statusfield 202, a license key status field 203 and a size field 204. Thus, asper FIG. 2, for example, a first entry of the disk management table 191indicates in the device number field 201 a disk drive 1, in theinstallation status field 202 that disk drive 1 is installed, in thelicense key status field 203 that the license key for disk drive 1 isinstalled, and in the size field 204 that the size of disk drive 1 is500 Giga bytes. Another entry of disk management table 191, for examplethe third entry, is similar to the first entry with the exception thatthe third entry concerns disk drive 3, that the license key for diskdrive 3 is not installed and that the size of the disk drive 3 is 250Giga bytes. Yet another entry of disk management table 191, for examplethe fifth entry, is different from the other entries in that the fifthentry concerns disk drive 5 and that disk drive 5 is not installed.According to the present invention a license key must be installed inorder to use a corresponding disk drive and of course a disk drive cannot be used if it has not been installed.

Thus, according to the present invention disk management table 191indicates whether or not a disk drive has been installed, whether or nota license key for the disk drive has been installed and the size of thedisk drive if installed. Therefore, the disk management table 191 tracksinformation regarding the status of respective disk drives, therebypermitting decisions to be made so as to properly manage the resourcesof the storage system particularly with regard to the queuing area 196.

FIG. 3 illustrates an example of a cache management table 190 formanaging cache memory to implement the present invention. As per FIG. 3,the cache management table 190 includes a plurality of entries, whereineach entry includes at least a slot number field 301, an installationstatus field 302, a license key status field 303 and a size field 304.Thus, as per FIG. 3, for example, a first entry of the cache managementtable 190 indicates in the slot number field 301 a memory device 1, inthe installation status field 302 that memory device 1 is installed, inthe license key status field 303 that the license key for memory device1 is installed, and in the size field 304 that the size of memory device1 is 2 Giga bytes. Another entry of cache management table 190, forexample the second entry, is similar to the first entry with theexception that the second entry concerns memory device 2, that thelicense key for memory device 2 is not installed and that the size ofthe memory device 2 is 2 Giga bytes. Yet another entry of cachemanagement table 190, for example the third entry, is different from theother entries in that the third entry concerns memory device 3 and thatmemory device 3 is not installed. According to the present invention alicense key must be installed in order to use a corresponding memorydevice and of course a memory device can not be used if it has not beeninstalled.

Thus, according to the present invention cache management table 190indicates whether or not a memory device has been installed, whether ornot a license key for the memory device has been installed and the sizeof the memory device if installed. Therefore, the cache management table190 tracks information regarding the status of respective memorydevices, thereby permitting decisions to be made so as to properlymanage the resources of the storage system particularly with regard tothe queuing area 196.

FIG. 4 illustrates an example of a monitoring results table 400 forstoring the results of monitoring the resources of a storage systemparticularly with regard to the queuing area 196 according to thepresent invention. The monitoring results table 400, for example, isstored in either the storage 198 of the management center system 150 orin the resource/license manager 195 included in each of the storagesubsystems 101 and 102. The monitoring results table 400 includes aplurality of entries wherein each entry includes a device identification(ID) field 401 for identifying a device number of a device being used inthe queuing area 196, a device field 402 which identifies the type ofdevice either a cache or disk drive being used in the queuing area 196,a configuration field 403 for indicating an amount of the total capacityof the queuing area the device represents according to the particularconfiguration, a capacity field 404 which indicates the capacity of thedevice and a current usage field 405 for indicating the amount of thecapacity of the device being used.

Thus, for example, as illustrated in FIG. 4, the first entry of themonitoring results table 400 in the device ID field 401 identifiesdevice 1, in the device field 402 identifies device 1 as a cache, inconfiguration field 403 indicates that the cache of device 1 represent70% of the total capacity available, in the capacity field 404 indicatesdevice 1 has a capacity of 32 Giga bytes, and in the current usage fieldindicates the current usage of the cache amounts to 30% of the availablecapacity.

As per the present invention, the management center system 150 or theresource/license manager 195 makes use of the information stored in themonitoring results table 400 so as to determine whether an action shouldbe taken or not. Also, the monitoring results table 400 can be used topermit historical analysis of the usage made of the devices forming thequeuing area 196 so as to determine whether a particular trend isexhibited to which a counter process or procedure can be implemented.

FIG. 5 illustrates a flowchart of an example of a process executed byeither the management center system 150 or the storage subsystems 101 or102. It should be further noted that each of the steps illustrated inthe flowchart of FIG. 5 can, for example, be implemented by one or moreinstructions of a computer program stored in a storage medium such asthe hard drive of a computer system, a floppy disk, CD ROM, electricallyerasable programmable read only memory (EEPROM), etc.

As per FIG. 5, the process starts with the user defining, setting theevent or condition to be detected and the specific action to beperformed when the event or condition has been detected (step 501). Theevent or condition to be detected could, for example, include detectingwhen the current usage of the queuing area 196 exceeds a predeterminedlimit. This limit could be set at any level (60%, 80%, etc.) so long asit is appropriate to ensure efficient operation of the storage system.Information regarding the set event or condition is periodically checked(monitored) so as to detect whether the event or condition has occurred(step 502). The information to be checked could, for example, includethe entries in the monitoring results table 400. If the event orcondition has not occurred then the system continues monitoring theinformation (step 505). If the event or condition has occurred then theaction set by the user corresponding to the event or condition isperformed (step 503). The details of step 503 are illustrated in FIG. 6and will be discussed below. After the preset action has been performeda result of the action is output, for example, in the form of a noticeor charge (step 504).

As per the above, FIG. 6 illustrates a flowchart of an example of thedetails of processing performed to implement step 503 of FIG. 5. As perFIG. 6 in the action of step 503 the preset user defined actionincluding, for example, the management center alerting the user that thepreset event or condition has occurred by email, pager, etc., isperformed (step 5031). Thereafter, it is determined whether theadditional capacity (memory) needed by the queuing area 196 is to beautomatically provisioned or not (step 5032). If the additional memoryis not to be automatically provisioned then the system waits for aconfirmation as to whether the user defined action is to be conducted ornot (steps 5033 and 5034). This confirmation could, for example, beprovided by the management center system 150 or the user of the storagesystem being monitored. If the confirmation has not been received thenstep 5033 is conducted again. If the confirmation has been received thena new license key is sent to the storage subsystem requiring an increasein capacity of its queuing area 196 or the license key is removed fromthe storage subsystem (step 5035). Information indicating a result ofthe action taken, for example, installing a disk drive to the queuingarea is sent to either the resource/license manager 195 or themanagement center system 150 (step 5036). Thereafter, the informationindicating the result sent to either the resource/license manager 195 ofthe management center system 150 is stored in the monitoring resulttable 400 (step 5037).

It should be noted that step 503 of FIG. 5, the details of which areillustrated in FIG. 6 could, for example, be performed by the managementcenter system 150 while steps 501, 502, 504 and 505 could, for example,be performed in either the management center system 150 or a storagesubsystem 101 or 102. Further, it should be noted that the storage 198could, for example, also serve as a repository of collected informationincluding, for example, customer definition regarding features of thecustomer's storage subsystems and location information regarding thelocation of the customer's storage subsystems. This information isconsulted as needed by the management center system 150 when automaticprovisioning is not performed. In such a situation, a service personreceives such information from the management center system by email,pager or the like and uses such information to service the customer'sstorage subsystem at the customer's location upon detecting or beinggiven notice that an unused area of the queuing area is less than apredetermined level.

Therefore, based on the above, the present invention provides a methodand apparatus for increasing an amount of memory on demand in a storagesystem implementing a remote mirroring function. According to thepresent invention, the storage system includes first and second storagesubsystems which are connected to each other via a path wherein thefirst storage subsystem is connected to a host. As per the presentinvention, a primary volume is provided to the first storage subsystemand remote secondary volume is provided to the second storage subsystem,the remote secondary volume being a copied volume of the primary volumewhich is operated in a synchronous mode with respect to the remote thesecondary volume. As per the present invention, a memory such as cachememory, disk drives, memory devices or the like are provided in thesecond storage subsystem for temporarily storing data transferred to thesecond storage subsystem from the first storage subsystem in response toa write I/O issued by the host to write data in the primary volume. Datatemporarily stored in the memory is retrieved and stored in the remotesecondary volume. However, unique according to the present invention isthat an unused area of the memory is monitored and the amount of thememory is increased if the unused area becomes less than a predeterminedamount. Thus, the present invention provides for quick and efficientprovisioning of additional memory in the queuing area of a storagesystem on demand.

While the invention has been described in terms of its preferredembodiments, it should be understood that numerous modifications may bemade thereto without departing from the spirit and scope of the presentinvention. It is intended that all such modifications fall within thescope of the appended claims.

1. A method of increasing an amount of memory in a queuing area ondemand in a storage system implementing a remote mirroring function,said storage system including first and second storage subsystems whichare connected to each other via a path and the first storage subsystembeing connected to a host, the method comprising the steps of: providinga primary volume to the first storage subsystem and a remote secondaryvolume to the second storage subsystem, the remote secondary volumebeing a copied volume of the primary volume, the primary volume and theremote secondary volume being operated in an asynchronous mode;providing a first memory in a queuing area of the first storagesubsystem and a second memory in a queuing area of the second storagesubsystem for temporarily storing data transferred from the firststorage subsystem to the second storage subsystem in response to a writeinput/output (I/O) issued by the host to write data in the primaryvolume; retrieving data temporarily stored in the second memory andstoring the data in the remote secondary volume; and monitoring anunused area of the queuing area in at least one of the first storagesystem and the second storage system, and increasing an amount of atleast one of the first memory and the second memory therein when theunused area becomes less than a predetermined amount.
 2. The methodaccording to claim 1, wherein said increasing an amount of at least oneof the first memory and the second memory in the queuing area includesautomatically provisioning more memory to the queuing area.
 3. Themethod according to claim 2, wherein said automatically provisioningmore memory includes adding memory units to said queuing area.
 4. Themethod according to claim 3, wherein said memory units each includes atleast one of a cache memory and a disk drive.
 5. The method according toclaim 1, wherein said monitoring is performed by a resource/licensemanagement unit which monitors information collected with respect to theunused area of at least one of the first memory and the second memoryforming the queuing area.
 6. The method according to claim 5, whereinsaid queuing area is formed by a cache memory having at least one memorydevice and at least one disk drive.
 7. The method according to claim 6,wherein said information collected with respect to the unused area of atleast one of the first memory and the second memory forming the queuingarea includes information regarding the amount of use of the capacity ofsaid cache memory and information regarding the amount of use of thecapacity of said at least one disk drive.
 8. The method according toclaim 7, wherein increasing an amount of at least one of the firstmemory and the second memory forming the queuing area includes adding amemory device to said cache memory and/or adding another disk drive tosaid at least one disk drive.
 9. The method according to claim 7,wherein said information regarding the amount of use of the capacity ofsaid cache memory is provided by a cache management table having aplurality of entries each including at least a slot number field whichindicates a number identifying a memory device included in the cachememory, an installation status field which indicates whether the memorydevice has been installed, a license key status field which indicateswhether a license for the memory device has been installed and a sizefield which indicates the capacity of the memory device and wherein saidresource/license management unit monitors said cache management table.10. The method according to claim 7, wherein said information regardingthe amount of use of the capacity of said at least one disk drive isprovided by a disk management table having a plurality of entries eachincluding at least a device number field which indicates a numberidentifying a disk drive included in the at least one disk drive, aninstallation status field which indicates whether the disk drive hasbeen installed, a license key status field which indicates whether alicense for the disk drive has been installed and a size field whichindicates the capacity of the disk drive and wherein saidresource/license management unit monitors said disk management table.11. The method according to claim 1, wherein said increasing an amountof the memory in the queuing area includes notifying a management centersystem when the unused area becomes less than a predetermined amount,and wherein said management center system causes the amount of thememory of the queuing area to be increased.
 12. The method according toclaim 11, wherein said management center system causes the amount of thememory of the queuing area to be increased by notifying the user or aservice person that memory is to be added to the queuing area.
 13. Themethod according to claim 12, wherein said management center systemincludes a repository of collected information including, for example,customer definition and location information regarding the features andlocation of the customer's storage subsystems for use by the user or aservice person to add memory to the queuing area.
 14. A storage systemcomprising: a first storage apparatus which is connected to a host andincludes a primary volume and a queuing area having memory fortemporarily storing data transferred from the first storage subsystem tothe second storage subsystem in response to a write input/output (I/O)issued by the host to write data in the primary volume; and a secondstorage apparatus which is connected to said first storage apparatus viaa path and includes a remote secondary volume and a queuing area havingmemory for temporarily storing data transferred to the second storagesubsystem from the first storage subsystem in response to a writeinput/output (I/O) issued by the host to write data in the primaryvolume, wherein the remote secondary volume is a copied volume of theprimary volume and the primary volume and the remote secondary volumeare in asynchronous mode, wherein an unused area of the queuing area inat least one of the first storage apparatus and the second storageapparatus is monitored and an amount of the memory therein is increasedwhen the unused area becomes less than a predetermined amount.
 15. Thestorage system according to claim 14, wherein increasing an amount ofthe memory in the queuing area includes automatically provisioning morememory to the queuing area.
 16. The storage system according to claim15, wherein said automatically provisioning more memory includes addingmemory units to said queuing area.
 17. The storage system according toclaim 16, wherein said memory units each includes at least one of acache memory and a disk drive.
 18. The storage system according to claim14, wherein monitoring is performed by a resource/license managementunit which monitors information collected with respect to the unusedarea of the memory forming the queuing area.
 19. The storage systemaccording to claim 18, wherein said queuing area is formed by a cachememory having at least one memory device and at least one disk drive.20. The storage system according to claim 19, wherein said informationcollected with respect to the unused area of the memory forming thequeuing area includes information regarding the amount of use of thecapacity of said cache memory and information regarding the amount ofuse of the capacity of said at least one disk drive.
 21. The storagesystem according to claim 20, wherein increasing an amount of the memoryforming the queuing area includes adding a memory device to said cachememory and/or adding another disk drive to said at least one disk drive.22. The storage system according to claim 20, wherein said informationregarding the amount of use of the capacity of said cache memory isprovided by a cache management table having a plurality of entries eachincluding at least a slot number field which indicates a numberidentifying a memory device included in the cache memory, aninstallation status field which indicates whether the memory device hasbeen installed, a license key status field which indicates whether alicense for the memory device has been installed and a size field whichindicates the capacity of the memory device and wherein saidresource/license management unit monitors said cache management table.23. The storage system according to claim 20, wherein said informationregarding the amount of use of the capacity of said at least one diskdrive is provided by a disk management table having a plurality ofentries each including at least a device number field which indicates anumber identifying a disk drive included in the at least one disk drive,an installation status field which indicates whether the disk drive hasbeen installed, a license key status field which indicates whether alicense for the disk drive has been installed and a size field whichindicates the capacity of the disk drive and wherein saidresource/license management unit monitors said disk management table.24. The storage system according to claim 14, wherein increasing anamount of the memory in the queuing area includes notifying a managementcenter system when the unused area becomes less than a predeterminedamount, and wherein said management center system causes the amount ofthe memory of the queuing area to be increased.
 25. The storage systemaccording to claim 24, wherein said management center system causes theamount of the memory of the queuing area to be increased by notifyingthe user or a service person that memory is to be added to the queuingarea.
 26. The storage system according to claim 25, wherein saidmanagement center system includes a repository of collected informationincluding, for example, customer definition and location informationregarding the features and location of the customer's storage subsystemsfor use by the user or a service person to add memory to the queuingarea.